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Volume I: Technical Assessment Report 

1.0 Notification and Authorization 

The NASA Engineering and Safety Center (NESC) was requested to improve on the Best 
Practices document produced for the NESC assessment 07-001-E, Verification of Probabilistic 
Requirements for the Constellation Program (CxP), by giving a recommended procedure for 
using acceptance sampling by variables techniques. This recommended procedure would be 
used as an alternative to the potentially resource-intensive acceptance sampling by attributes 
method given in the document. 

An NESC out-of-board activity was approved on January 9, 2008. Mr. Kenneth Johnson at 
Marshall Space Flight Center (MSFC) was selected to lead this assessment. The assessment plan 
was approved at the NESC Review Board (NRB) on March 20, 2008. 
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4.0 Abstract 

Acceptance sampling is a method for verifying quality or performance requirements using 
sample data. Acceptance sampling by variables (ASV) is an alternative to acceptance sampling 
by attributes (ASA). In many instances, ASV requires significantly smaller samples than ASA. 

In an effort to make ASV more widely accessible, the team developed calculators for existing 
sampling plans identified in the open literature. For this paper, the literature on ASV was 
consolidated by providing a unified exposition of the approach used to develop such plans. From 
within this framework, the derivation of plans for exponential, normal, gamma, Weibull, inverse- 
Gaussian, and Poisson random variables were reviewed and verified. Verification unexpectedly 
surfaced a flaw in a published plan for inverse-Gaussian variables. A companion paper 
confirmed these results empirically and addressed the practical application of ASV when 
distributional forms were themselves uncertain. 
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5.0 Introduction 

One of the oldest problems in quality engineering is to assess the acceptability of items that a 
customer receives from a producer. Acceptance sampling is an alternative to 100-percent 
inspection applied when inspection is destructive, or when the time and/or cost of 100-percent 
inspection are unwarranted or prohibitive. The customer decides the disposition of an incoming 
lot based on a standard specifying the minimum proportion of nonconforming items in the 
sample. The decision can be to accept or reject the entire lot, or to continue sampling. 

Acceptance sampling also has been adapted to problems not identified with procurement. 

Smith et al. (2003) advocated the application of acceptance sampling in the context of water 
quality assessment. Bayard et al. (2007) applied the underlying concept to the assessment of 
future aircraft runway-incursion controls. White, et al. (2009) showed that acceptance sampling 
can be applied to any sampling experiment, including those commonly employed to verify 
design requirements using simulation and Monte Carlo methods. 

ASA determines the acceptability of a lot based on a count of the number of nonconforming 
items relative to the size of a random sample drawn from this lot. The inspection variable is 
binary (pass/fail) and therefore the count is necessarily a binomial random variable. ASA can be 
used with categorical outputs, or with outputs measured on a continuous or discrete scale, by 
reference to a required limiting value. Conceptually simple, easily applied, and universally 
applicable, ASA is the first choice for sampling inspection. 

ASV is an alternate approach, which in many instances prescribes significantly smaller samples 
than ASA. ASV requires that the inspection variable is measured on a continuous or discrete 
scale, that the distribution of this variable is known a priori and stable, and that a plan exists for 
this particular distribution. While far more restrictive in its assumptions, ASV may be 
considered when the assumptions are appropriate and the larger samples required by ASA are 
unavailable. 

White, et al. (2009) describes the development of ASA plans and provides an example 
illustrating the application of ASV to a Level 2 requirement from the Constellation Program 
(CxP). The first objective of this paper was to consolidate the scholarly literature on ASV and 
provide a readable tutorial on the concept and general approach used to develop ASV plans for 
alternative parent distributions. While ASV does employ a consistent procedure for determining 
sampling plans, this commonality was obscured by different presentation styles and the vastly 
different notation adopted by researchers in the field. This paper presents a unified presentation 
and a consistent notation. An example is provided in Appendix A. 

The second objective was to validate the mathematical derivations. Validation was successful in 
all but one case. A subtle flaw in the published plan for inverse-Gaussian variables was 
discovered that rendered it unusable, at least in its present form. A companion paper confirmed 
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these results and addressed the practical application of ASV when distributional forms were 
themselves uncertain. 

6.0 Probabilistic Requirements and Limit Standards 

In uncertain environments, requirements verification seeks to determine whether a measureable 
quantity is conforming or nonconforming, i.e., to determine whether or not the parent population 
from which a sample is drawn achieves a specified level of quality or performance. It is 
important to remember that this is a simplified and incomplete view of verification. A true 
requirement verification using these methods alone would be incomplete — a requirement can be 
judged as verified when a full understanding of the assumptions is clearly presented and 
understood. NASA-STD-7009, Standard for Models and Simulations (2008), offers a 
framework for doing much of this. The discussion on epistemic versus aleatory uncertainties in 
the Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners 
Version 1.1 (2002) 1 is also recommended. It is important to recognize that this verification 
process does not generally apply at all to the “Probable Risk Assessment-type” requirements that 
directly address loss of crew or vehicle. 

The remainder of the document will address this partial verification. The methods outlined in 
this paper were meant to improve the rigor of the portion of the requirements verification process 
involving the raw exercise of a simulation model and the comparison of its output to the stated 
quantified requirement threshold. These methods do not address the critical wider issue of 
analysis method assumptions. 

A probabilistic requirement can be stated probabilistically as a (/, p , fJ) limit standard 
(White, et al., 2009), where: 

(1) /is the performance indicator. The measured quantity may be inherently categorical or 
qualitative in nature and performance indicated by occurrence or nonoccurrence of some 
event. Alternately, the measured quantity may be inherently quantitative and 
performance is indicated by success or failure in achieving a limit or tolerance. 

(2) p is the minimum reliability for the population. This is the minimum, acceptable, long- 
run proportion of observations on which the population achieves the desired performance. 
If p is the failure probability for the population, then the requirement is p < 1 - p. 


1 Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners Version 1.1 has been 
replaced by NASA/SP-201 1-3421 Probabilistic Risk Assessment Procedures Guide for NASA Managers and 
Practitioners second edition. 
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(3) ft is the maximum acceptable consumer’s risk. This is the probability of incorrectly 
accepting a nonconforming population as the result of sampling instead of 100-percent 
inspection. 

Formally, a measurable quantity is represented as the random variable X with a probability 
distribution of F{x;9) and density \sf(x;0). where 0 is an unknown distribution parameter. Let 
{ Xi ; i = 1, ..., n} be a random sample with observed values {xg i = 1, ..., n}. The verification 
problem is to determine whether the population as a whole conforms to a specified limit 
standard, based on the statistics of the sample observed. 

7.0 Sampling Plans and Operating Characteristics 

A sampling plan is the pair ( n,K ), where n is the minimum sample size, i.e., the minimum 
number of observations required to verify statistically the requirement imposed by the standard. 

/r is a constant factor which is used to assess whether or not the population is conforming. The 
interpretation of k depends on whether the characteristic is continuous or discrete, as discussed 
in Section 8.0. 

For a given distribution, every sampling plan has a unique operating characteristic (OC). The 
OC is a function that defines the probability of accepting a population, P a , for every value of the 
failure probability, p, i.e., the OC is the function PJp) where p e [0, 1 ] . A sampling plan is 
derived by first defining two operating points , (po, 1 - a) and (p\,p), where po < pi and a and [i 
are small probabilities. It is then required that OC must (minimally) satisfy the following 
inequalities: 

P a (po ) > 1 - a (Eq. 1) 

P a (p i)<P- (Eq. 2) 

Alternately, it is sometimes more convenient to express these in terms of the rejection 
probabilities as the power conditions P r (po) = 1 - P a (po ) < cc and P,(p i) = 1 - P a (p 0 > 1 - (:’>)■ 
Note that when p\ = 1 -p and /? is the consumer’s risk, inequality (Eq. 2) enforces the limit 
standard. Under this plan, a population with failure probability p\ as conforming to small 
probability /3 is noted. Inequality (Eq. 1) captures the competing good. Under this plan, a 
population with probability /; (l as conforming to high probability 1 - a is noted. 

The probability of incorrectly rejecting a conforming system, a , was called the producer’s risk. 
When given as percentages, 100-percent x (1 - po) was called the acceptable quality level and 
100-percent x (1 - p\) was called the lot tolerance percent defective. Note that the limit standard 
does not specify the (po,l - a) operating point. It was common to develop a range of plans with 
the required consumer’s risk y 3 and differing^:. Plans with larger sample sizes n then have 
smaller corresponding a. 
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8.0 Hypothesis Testing and Acceptance Limits 

The underlying problem was framed as a hypothesis test, which was intended to enforce both 
significance and power requirements. The null and alternate hypotheses are: 

Ho: p=po and Hp. p=p\> po 

Under Ho, the population was accepted as conforming and under H\ the population was rejected 
as nonconforming. Inequality (Eq. 1) establishes the significance of test as a and inequality 
(Eq. 2) establishes the power of the test as 1 - J3, where a and ft are the probabilities of a Type I 
error (the producer’s risk or the risk of rejecting a population that should have been accepted), 
and a Type II error (the consumer’s risk, or the risk of accepting a population that should have 
been rejected), respectively. 

The sample data were used to choose between the null and alternate hypotheses. To accomplish 
this, the critical value of an appropriate test statistic needed to be determined. Denote the test 
statistic as the acceptance limit A(n,tc), where the arguments are the parameters of the sampling 
plan. For a continuous quantity, k = k is the acceptance limit and typically has the form: 

A(n, ±k) = 0(n) ± k<j{n) 


where the plus is used with an upper bound and the minus with a lower bound. It follows that: 


p _ A(n, k) - Q(ri) 
&{n) 


(Eq. 3) 


where 6 and cr are estimators for the unknown parameter and the standard deviation of the 
population. 

For I a lower specification limit with value x mm , the desired performance on the z th observation is 
achieved if and only if x\ > x mm . For the sample as a whole, the null hypothesis is rejected if 
A(n,—k ) < x mm . That is, the acceptance limit is smaller than the specified limit. 

For I an upper specification limit with value x max , the desired performance on the z th observation 
is achieved if and only if x-, < x max . For the sample as a whole, the null hypothesis is rejected if 
Ain, -k) > x max . That is, the acceptance limit is larger than the specified limit. 

The effect in either case is to move the critical point of the acceptance limit away from the 
estimate of the unknown quantity in the direction of the specified limit by k standard deviations. 
Intuitively, this hedge is intended to compensate for error in the estimated mean for a sample of 
n observations in a statistically exact way. 

Note that in this paper the third case will not be addressed, where two limits are specified. In 
this case, / is a tolerance interval, x max > x, > x m /„. The derivation of sampling plans for tolerance 
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is modestly more complicated than that described in Section 8.0. The reader is referred to the 
literature on tolerance intervals. 

For a discrete population characteristic, the acceptance limit has the form: 

A(n,c) = Y(n) — c 

where Y is some function of X with nonnegative integer values. It follows that: 

c = Y(n)-A(n,c). (Eq. 4) 

For a lower specification limit on Y, the acceptance number, c, is the minimum acceptable value 
of Y{n) and the rejection criteria is A(n,c) < 0. For an upper specification limit on Y, c is the 
maximum acceptable value of Y{n) and the rejection criteria is A(n,c) > 0. 

9.0 General Procedure for Developing Variables Acceptance 
Sampling Plans 

For a continuous distribution with lower specification limit, the procedure comprises 4 steps: 

(1) Determine the value of ft (typically the mean) for the null probability distribution with 
failure probability po = Pr[X < x nun \ = F(x m i n ; ft); determine the value of ft for the 
alternate distribution with failure probability p\ = Pr[A < x min . \ = F(x mjn ;6\). This is 
accomplished using the inverse distribution. As shown in Figure 9.0-1, the requirement 
that po < p i implies that ft) > ft. 

(2) Determine the minimum value of n from the sampling distributions for ft 0 and ft) such 
that both inequalities (Eq. 1) and (Eq. 2) are satisfied. 

(3) Determine the maximum acceptance limit, Aq, from the null sampling distribution with 
a = Pr [ft < ft,] = F(ft; 0 [V n ) ; determine the minimum acceptance limit, A i, from the 
alternate sampling distribution with ft = Pr[ft — A] = F(ft; ft, , n ) ; as illustrated in 
Figure 9.0-2. 

(4) Determine the factors ft and ft corresponding to Aq and A | , respectively, from 
equation (3). In general, these factors are not equal. A conservative choice is to use the 
larger of the two values as the factor k for the sampling plan. Alternately, k is sometimes 
taken as the average of these two values. 

For 7, an upper specification limit, the failure probability of the population is 
p = Pr( A > x max ] = 1 - F(x;0 ). The procedure is identical with this single exception. For a 
discrete distribution, instead of the factors ft) and ft, at Step 4 the constant c is determined. 
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Figure 9.0-1. Distribution Functions and Critical Values for X Under the Null and Alternate 

Hypotheses 



Figure 9. 0-2. Sampling Distributions for the Estimator 6 Under the Null and Alternate 

Hypotheses 

If the random variable X and the unknown parameter 0 can be standardized, it is generally more 
convenient to use the standardized distribution and standardized sampling distribution to derive 
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sampling plans. This convenience is illustrated in Section 10 for exponential and normal random 
variables. 


10.0 Methods Implemented 

A literature search found derivations of ( n,k ) sampling plans for exponential, normal, gamma, 
Weibull, and inverse-Gaussian distributions. These derivations follow the general procedure 
given in Section 9.0. A derivation was also found for the ( n,c ) sampling plan for the Poisson 
distribution. In this section, these derivations are outlined with reference to lower specification 
limits. The corresponding derivations for upper specification limits are easily deduced. 

10.1 Exponential 

Guenther (1977) considers the exponential random variable X with unknown mean //, supports 
xe[0,oo), and distribution function: 

F(x;p) = l-e- xlM . 

Note that exponential and chi-squared distributions are both special cases of the gamma 
distribution, with exponential (//) = chi-squared(2) = gamma( 1 ,//). 

Denote a random variable distributed chi-squared with v degrees of freedom as Y V X can be 
standardized as: 



A 


so that p = Pr( A <x p ) = Pr(T < y 2 . p ) and 

_ 2 

y 2: n Anin 

A 


(Eq. 5) 


where p is either po or p \ . Similarly, the estimator for the mean // is sample mean X ( n ) , which 
can be standardized as: 


Y 1 .= — X(n). 


Substituting equation (5) into equation (6) yields: 


2 n 


y2n-, P =— x( F) = 


n y n - P - 


x(n) 


A 


where p is either p {) or p , . Inequalities (Eq. 1) and (Eq. 2) imply: 


(Eq. 6) 
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2 n 


y*.*> = 


= — x(n) = —^x(n) < y 2n . a 


M 


2 n y 2 ; Pl 

y 2 n-, P] = —x(n) = x(n) > y 2n ,^py 

V x^ n 


Together, these in turn imply that: 


yin\a > x(n) > y 2n -( 1 -fj) 

yi- Po X m [ n y 2\p x 


or 


> y^n-.a-p) 

y2',p 0 y 2n\a 


(Eq. 7) 


The minimum required sample size is the smallest value of the integer n satisfying inequality 
(Eq. 7), which can be determined by table lookup. 

With n known, the k-factor can be computed. The estimator for both the mean and standard 
deviation is the sample mean, X(n ) ; therefore, the acceptance limit is: 


A(n,—k ) = jj(n) — k'a(n) = (1 — k)X(n) = (1 — k') — Y 2n . 

2 n 

It is customary to let k = 1 —k’ and use the k-factors and 


k 


0 


;po 

y 2 n;a 


K 


R y 2;a 

y 2 n;(l-P) 


10.2 Normal (a-known) 

The derivation of sampling plans for measureable quantities distributed N(//,cr 2 ) is 
widely published (Bowker and Goode, 1952; Guenther, 1977; Kao, 1971; Lieberman and 
Resnikoff, 1955; and Montgomery, 2005; among others) and the basis for standards 
Military Standard (MIL-STD)-414 and American National Standards Institute/ American Society 
of Quality Control Z1.9, and International Organization for Standardization 2859. Consider the 
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normal random variable X with unknown mean /u, known standard deviation cr, supports 
jte (- 00 , 00 ), and density function: 

f(x;ju,a)= 1 


\j 2/TfT ’ 

Denote a random variable distributed a standard normal N(0,1) as Z. X can be standardized as: 


such that p = Pr(X < x p ) = Pr(Z < z p )is: 




A — x(n ) 


(Eq. 8) 


The mean estimator X(ri) is distributed N (ju,cf/n), which can be standardized as: 

- X(n)-p 
Z(n) = — 


cr \jn 


(Eq. 9) 


where Z{n) also is a standard normal N(0,1) deviate. 

The acceptance criterion is A(n, -k) = x(n) -k<7> x mm , so that the population is rejected if: 


< k . 


Inequalities (Eq. 1) and (Eq. 2) imply: 


Pr 


X(n)-x B 


< k 


a 


= Pr 

= Pr 

= Pr 
= Pr 


X{n)-x n 


c>Mn 


< k\In 




cr/ yjn <7 sjn 
Z- z p sfn < k sfn 


Z <4n(k + Zp) 


The acceptance-limit criteria (expressed as power requirements) from inequalities (Eq. 1) and 
(Eq. 2) then imply: 


n(k + z Po )>z a 
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4n(k + z Pi )> z (HJ) = z x _ p = -z p . 

Together, these imply that: 


and therefore 



K 



n > 


Z a +Z P 

z — z 

. ^Pi ^ Po 


(Eq. 10) 


(Eq. 11) 


The minimum required sample size is the smallest integer value of n satisfying inequality 
(Eq. 11). With n known, the k-factors can be computed as from equations (Eq. 10). 

10.3 Normal (cr-un known) 

The more usual case is where cr is unknown and must be estimated from the sample data. While 
an exact approach is available using a non-central /-distribution (see Guenther, 1977, among 
others). For modest sample sizes, excellent approximations for n and k are achieved applying a 
result by Cramer (1945). Let X(n) and S(n) be sample mean and standard deviation, 
respectively. For n sufficiently large, the distribution of the acceptance limit 
A(n,k ) = X(n)-kS(n) is asymptotically normally distributed N (jUa,cTa), with mean 


M A =M±kc> 


and variance: 


cr: 


n 


-e(fc, £>,&)• 


2 

The limiting variance cT/n is weighted by the expansion factor e(k,% 3 ,^ 4 ) (Takagi, 1972). For a 
non-Normal random variable X with mean // , standard deviation cr, skew £ 3 , and kurtosis £ 4 , 


e(k,£,£ 4 ) 


i+E(£ 4 -i )±<4 


(Eq. 12) 
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where the plus is used with an upper bound and the minus with a lower bound. For a Normal 
random variable X, the skew is £3 = 0 and the kurtosis is Ca = 3. The expansion factor therefore 
reduces to (Wallis, 1947): 


e(k, 0,3) 




The derivation of n and k then proceeds as in the cr-known case, resulting in: 

-\2 

z„+z„ 

n>e(k, 0,3) p 


z — z 

Pi Po 


and 


^ _ _ z a z p t + z p z Po 

Z a +Z fl 

The minimum required sample size is the smallest integer value of n satisfying the inequality; 
with n known, the //-factors can be computed. 

10.4 Gamma 


Takagi (1972) considers the gamma random variable X with unknown location 8<_x < 00 and 
estimated shape v> 0 and scale A>0 (scale) parameters, support ve[<7,oc), and density function: 


f(x;v,A,8) 


{(x-s)ny-' t 
/tny) 


The moments for this distribution are: 

/j = 8 + uA, 
a 2 = vX 1 

ft = 2 /^ < E£ 1- 13 ) 

ft =3+6/1/. 

With the same normal approximation N(f.i,\,cf A ) for A(kji), the expansion factor given in 
equation (12) is computed using the gamma moments in equations (13). The derivation is 
essentially the same as that for the normal distribution, resulting in: 
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n>e(k,^ 3 ,^ 4 ) 


-|2 


Z a Z [S 

t ~t 

Pi Po . 


and 


k = ■ 


Z J Pl + Vpo 

Z a + Z fi 


Here, 


t = ■ 


X-S 


is a standard gamma deviate with density: 


V — 1 


f(t,v) = - <T f 

r(v) 


(Eq. 14) 


(Eq. 15) 


for which /jt = 0 and Sr = 1 . 

The minimum required sample size is the smallest integer value of n satisfying the inequality; 
with n known, the k-factors can be computed. 

10.5 Weibull 


Takagi (1972) also considers the Weibull random variable X with unknown location S<x < oo 
and estimated shape v> 0 and scale A, > 0 (scale) parameters, support .re[c>,oo), and density 
function: 


f(x\v,A,S) 


x-5 

A, 


Jci x ~ s )I x J ' 


The moments for this distribution are: 


ju = S + A.fi T 
a 1 = Arc> 2 T 

£ = [E(3Zz + 1) - 3T(2 b + 1) + 2T 3 (b + l)]/cr 3 
£ = [r(4Z? + 1) - 4r(3Z? + l)T9b + 1) + 6T(2b + V)T 2 (b + l)]/<7* . 

Here, 


(Eq. 16) 
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T _ (x-r) 

i 

is a standard Weibull random variable with density: 

f{t\v) = vt v ~ l e ,v 

and mean and variance: 

jj. T =F(b + Y) 
a~ =Y{2b + l)-T\b + l) 


where b = \/v. 

The expansion factor in (Eq. 12) is computed using the Weibull moments in (Eq. 16). The 
expressions for n and k are again given by (Eq. 14) and (Eq. 15), where it is understood that now 
t is the Weibull deviate. 

10.6 Inverse Gaussian 

Aminzadeh (1996) considers the inverse-Gaussian random variable X with unknown location 
parameter 8 <x < go, estimated mean ju> 0 and shape parameter 2 > 0, support re [0,oo), and 
distribution function: 


where: 


F(x\ju,A) = F 1 (x",/i,A) + F 2 (x',jU,A) 


F l (r,/i,A) = <p 


21 / 


(x; ju,A)-e /u (p 




\ 


--1 


Vf ), 

/ 

X 

( 

X 



— + 1 

X 


)) 


and (jF) is the distribution function of the standard normal distribution N(0,1). As explained 
below, the derivation of the inverse-Gaussian sampling plan appears to be flawed. 

As X tends to infinity, the inverse-Gaussian distribution F(x;jli, A) is asymptotically normal with 
distribution F l (x; J u,X). On this basis, it is argued that if the shape-to-mean ratio is large 
{AJ/u > 10), then F 2 (x;ju,A) is almost zero the failure probability of p = F i(x;/j,X). This 
relationship is used to determine the means for the null and alternative distributions as described 
in Section 9.0. 
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While it is true that for F 2 (x;p A) is almost zero when A/p > 10, it is also true that F\{x;/u,X) is 
almost zero. Thus, the second term cannot simply be ignored in the calculation of po and p\. 

This is illustrated in Figures 10.0-1 and 10.0-2 for AJp = 10 and lower limit A' mm e[0,//|. 

Figure 10.0-1 shows the values of Fi(x;p,A) and F 2 (x;p,A) as functions of the limit-to-mean ratio 
x m J pe[Q,\\- Also shown is the sum and ratio of these two terms, p = F 2 {x;p,A) + F 2 (x;p, 2) and 
R = F 2 (x;p, A)/F\{x;p, 2) . Clearly, the contribution of F 2 {x;p,A) in the computation of p is 
nontrivial for the entire range of lower limit. For the small failure probabilities typically of 
interest in acceptance sampling, the contribution of F 2 (x;p, AS) is nearly equal to that of F\{x;p,A). 

Figure 10.0-2 illustrates the dependency of ratio R on the shape parameter for selected values of 
AJp>_ 10, 20, and 100. Note that as A/p increases and the distribution becomes increasingly 
normal in shape, the contribution F 2 {x;p,A) in the computation of p diminishes, but remains 
nontrivial, especially for small failure probabilities. 





Figure 10.0-1. The Individual Terms of the Inverse-Gaussian Distribution, the Ratio 
R = F 2 (x;p,A)/F ,(x;p,A), and the Sum p = F,(x;p,A) + F 2 (x;p,A.), as Functions of the 
Lower-Limit-to-Mean Ratio x mi ,/p for the Shape-to-Mean Ratio A/p = 10 
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Figure 10.0-2. The Ratio of the Terms of the Inverse-Gaussian Distribution, 

R = F 2 (x;p,X)/F ,(x;p,X), and the Corresponding Failure Probability, p, as Functions of the 
Lower-Limit-to-Mean Ratio x min /p, Parameterized on the Shape-to-Mean Ratio for X/p = 10, 20, 100 

10.7 Poisson 

Guenther (1972, 1977) considers the Poisson random variable X with unknown mean ju, supports 
xe[0,l,2,...), and distribution function: 

F(x\p) = e~^^-, 

i=o /! 

The means po and p\ can be determined form the inverse distribution. Alternately, Guenther 
exploits the relationship between Poisson and chi-squared distributions and calculates the means 
using the expression: 

M 2 

Consider A as a discrete random variable representing the number of usable items in a lot and 
specification limit Xmin as the specified minimum number of usable items in each lot. If a sample 
of n lots are obtained, the total number of usable items obtained is the sum: 
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r =Y. x ,- 

7=0 

which is distributed Poisson with a mean of np, i.e., F(y; nju). Because the team is working with 
discrete random variables (as is the case in attributes acceptance sampling), the rejection 
criterion is based on the minimum total number of usable items in all of the lots, y > c + 1 = d, 
instead of the distance k. Inequalities (Eq. 1) and (Eq. 2) are: 

Paip >Po)= 1 - F(y d ] njuo) > 1 -a 

P a (p >Pi)= 1 - F(d\ np,) < p. 

Or, more simply, the power requirements F(d; rip i) >_1 - ft and F(d\ np {) ) < a. Again exploiting 
relationship between Poisson and chi-squared distributions, the power requirements imply: 

2 V 2 

^Cld\(x «■ «■ yC'2.d'\—p / t -1 i a\ 

>n>—^. (Eq. 14) 

2// 0 2//j 

This expression can be solved for a (d,n) sampling plan by enumeration. The value of integer 
values of d is increased until an integer value of n is found which satisfies both inequalities 
(Eq. 14). 

11.0 Conclusions 

The work reported in this paper is the first phase of a project intended to make ASV a practical 
alternative to ASA when appropriate variables plans are available. Published plans for 
exponential, normal, gamma, Weibull, and inverse-Gaussian random variables were discovered 
while the literature was reviewed. With the exception of normal plans, the search for off-the- 
shelf ASV plan calculators was fruitless. 

To perfect an understanding and to facilitate the future development of variable plans for a wider 
selection of distributions, a consistent notation was introduced and interpreted the procedure for 
developing plans within the common framework of hypothesis testing (following the lead of 
Guenther (1972, 1977)). The result presented here is a consolidation of the existing literature. 

In reviewing the methods presented, the mathematical derivations were verified. One 
unanticipated result was the discovery of a flaw in a published plan for inverse-Gaussian random 
variables that renders this plan unusable without modification. The flaw is subtle and this plan 
has remained unchallenged for over a decade. 

As reported in Volume II (Probabilistic Requirements (Partial) Verification Methods Best 
Practices Improvement), the completion of this foundational research allowed team members to 
implement plan calculators and test the accuracy of these plans empirically. Testing is necessary 
since, as this Volume has shown, the majority of variables plans are based on approximating the 
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standard deviation by employing an expansion factor to the sample standard deviation. The final 
phase of this project will address the practical application of ASV when distributional forms are 
themselves uncertain. The results of this additional research are reported in companion papers. 


12.0 Acronyms List 


ASA 

ASV 

ATK 

CxP 

LaRC 

MIL-STD 

MSFC 

MTSO 

NESC 

NRB 

OC 


Acceptance Sampling by Attributes 
Acceptance Sampling by Variables 
Alliant Techsystems, Inc. 

Constellation Program 
Langley Research Center 
Military Standard 
Marshall Space Flight Center 
Management Technical Support Office 
NASA Engineering and Safety Center 
NESC Review Board 
Operating Characteristic 
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Appendix A. Acceptances Sampling by Attributes 

A producer ships a large number of bolts built to withstand 500 foot-pounds of torque without 
breaking. The shipment comes in batches called lots, with 20,000 bolts/lot. Each lot will be 
inspected to determine if the bolts satisfy the minimum torque requirement. Inspection is 
destructive (a broken bolt cannot be used), so acceptance sampling must be applied. A sample of 
n bolts from each lot will be drawn at random and each bolt in the sample will be tested to see if 
it will withstand 500 foot-pounds of torque. If more than c bolts in the lot break, then the entire 
lot will be rejected. Otherwise, the entire lot will be accepted. 

The requirement is: 

(1) Performance indicator. An individual bolt is conforming if X >_500 foot-pounds, where 
X is applied torque at which the bolt breaks during testing. Otherwise, the bolt is 
nonconforming. 

(2) Minimum reliability. A lot is conforming if no more than 100 bolts (0.5 percent) in that 
lot are nonconforming. That is, the minimum reliability required is p = 0.995. If p is the 
failure probability for the lot, then requirement is p < 1 - p = 0.005. 

(3) Maximum acceptable consumer’s risk. To achieve the benefits of sampling, the 
consumer recognizes that occasionally a nonconforming lot will be accepted as 
conforming when it is not. The consumer requires that this happen on average no more 
than once for every twenty nonconforming received. That is, the maximum consumer’s 
risk is J3= 0.05. 

How large should each sample be? How many individual nonconforming bolts can be observed 
in the sample and still accept the entire lot as conforming to the requirement? 

Let Y{n) be the number of bolts in a sample of n bolts that fail. Since each sampled bolt will be 
broken, the sampling in this case is sampling without replacement. The distribution of Y{n) is 
therefore hypergeometric. However, if n is less than 10 percent of the lot size, then the binomial 
distribution is a good approximation. Assume therefore that Y{n) will have a binomial 
distribution with the probability of failure p. 

Figure 1 is a screen shot of the binomial (attributes) sampling plan calculator developed during 
this assessment. A minimum reliability of 0.995 and a consumer’s risk of 0.05 have been 
entered as inputs. The third input is the reliability at which the producer’s risk is calculated. The 
results shown provide twenty ( n,c ) sampling plans, parameterized on producer’s risk. For 
example, a (1549,4) plan is sufficient to test the requirement with an associated producer’s risk 
of a = 0.0385. That is, samples of 1549 bolts (7.774 percent of the 20,000 bolts in a lot) will be 
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drawn at random for testing. Lots with samples which include more than four nonconforming 
bolts will be accepted. Lots with a failure probability greater than 0.005 will be accepted of 
5 percent of the time on average. Lots with failure probability less than 0.999 will be rejected 
3.85 percent on average. Also shown are the 20 corresponding OC curves, which plot the 
acceptance probability as a function of the maximum failure probability for each plan. Note, that 
by design, all of the OC curves intersect at p = 0.005 and P a = 0.05. 


Dute (n, c) Acceptance Sampling Plans 

revKPW 17 June 2011 


ENTER the lower end upper test 
points ( reliabilites between 0 and 
0.999) and fixed (i risk. 



Input 

» r. 


0.995 0.05 0.999 



The corresponding (n, c) sampling plan 
and associated a risks are tabluated. 


Plan is calculated using a binomial distribution for the number 
accepted, with the correspoonding assumption of an infinite 
parent population. Calculator currently is limited to maximum 
n= 10,000 and c=20.) 


Operarting Characteristic (OC) Curves for all plans. 


Results 

c 

n 

a 


0 

598 

0.450254151 


1 

947 

0.244743107 


2 

1258 

0.133243744 


3 

1549 

0.071846749 


4 

1829 

0.03845934 


5 

2100 

0.020388648 


6 

2366 

0.010754564 


7 

2627 

0.005633657 


8 

2884 

0.002933176 


9 

3138 

0.001519926 


10 

3389 

0.000783379 


11 

3638 

0.000205865 


12 

3885 

0.000205865 


13 

4130 

0.000104904 


14 

4374 

5.33599E-05 


15 

4616 

2.70211 E-05 


16 

4857 

1 36549E-05 


17 

5096 

6.86761 E-06 


18 

5335 

3.4551 7E-06 


19 

5572 

1.72971 E-06 


20 

5808 

BJ3824E47 




1.0000 

0.9000 

0.8000 

0.7000 

0.6000 

0.5000 

0.4000 

0.3000 

0.2000 

0.1000 

0.0000 




Faflur* ProbatotMy, ,*> 





Figure 1. Screen Shot of the User Interface for the Binomial Sampling Plan Calculator 


Acceptance Sampling by Variables (ASV) 

The ASV plan requires destruction of a considerable fraction of the bolts in each lot (but, 
obviously, far less than 100-percent inspection). Suppose that, based on past experience, it can 
be assumed that the random breaking torques have a normal distribution with mean 
H = 528 and standard deviation cr= 10. If this assumption is can be sustained, then a sampling 
plan based on normal random variables should reduce the waste from inspection. Let the random 
variables X\, X 2 , .... X„ be a random sample n such measurements, with sample mean: 


X(n) = -±T I 


n , =( 


=0 
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and sample standard deviation: 


S(n) 


1 


! n(n - 1) 




How large should this sample be? How small can the sample mean be and still accept the entire 
lot as conforming to the requirement? 


Figure 2 is a screen shot of the normal sampling plan calculator developed during this 
assessment. The historical mean and standard deviation have been entered as inputs. The 
required reliability, consumer’s risk, maximum reliability, and lower limit on torque also have 
been entered as inputs. 


ENTER the parameters of the Normal(^,o.) distribution 
in the fields below. 


Input 

Mean u 

Std Dev a 

from fit 

528 

10 



variable X based on the requirement. 


Input 

P 

0 

P- 

A ... 

requirement 

0.995 

0.05 

0.999 

500 


If the limit Is less than the mean, the value entered is assumed to 
be a lower limit; If the limit is greater than the mean, the value 
entered is assumed to be an upper limit. 



Results 

(o estimated) 

(o Known) 

k 

a 

Accept 






0.001 

Yea 



373 


2.762898676 

0.002 

Yea 

527.63 


353 

73 

2.76845119 

0.003 

Yea 

527 68 


339 


2.772741736 

0.004 

Yea 

527.73 


327 


2.776298673 

0.005 


527.76 


318 


2.779369652 

0.006 

Yea 

527.79 


310 

64 

2.782092957 

0.007 

Yea 

527.82 


303 


2.78455299 

0.008 


527.85 


297 


2.786B0639 

0.009 

Yea 

527.87 


292 


2.7B6B92691 

0.01 


527.89 


256 


2.604596152 

0.02 

No 

528.05 


234 

47 

2.815818613 

3.03 


528.16 


218 

44 

2.625014329 

0.04 

No 

528.25 


206 

41 

2.833033805 

0.05 

No 

528.33 


195 

39 

2.840271873 

0.06 

No 

523.43 


166 

37 

2.646964634 

0.07 

No 

528.47 


179 

36 

2.85325173 

0.08 

No 

528.53 









166 


2.664961392 

0.1 

No 

528.65 


141 

26 

2.691393299 

0.15 


528.91 


123 


2.916117339 

0.2 

No 

529.16 










18 

2.965879335 

0.3 

No 

529.66 


86 


2.992600283 

0.35 

No 

529.93 



14 

3.021576495 

0.4 

No 

530.22 


66 


3.05372263 

0.45 

No 

530.54 


60 


3.090232306 

0.5 

No 

530.90 


DETERMINE If the design is acceptable from the plan with the largest value of n 
which is no greater than the number of data points used to fit the distribution. 
Note that the standard deviation almost certainly is astlmatmd from the data fit. 


Operating Characteristic (OC) or Power Curves for all plans. 



Calculation! for n are approximate for ^aatlmatad, using the Wallis formulas as reported by Duncan (1974) and 
Guenther (1977). 


Figure 2. Screen Shot of the User Interface for the Normal Sampling Plan Calculator 


The results shown provide 20 ( n,k ) sampling plans, parameterized on producer’s risk. For the 
same requirement, a (218, 2.283) normal plan provides comparable protection to the (1549,4) 
binomial plan examined in preceding example, with a producer’s risk of a = 0.040. Indeed, if 
the historical standard deviation is certain, then an even smaller (44, 2.283) plan will suffice. 
The significant reduction in sample size is apparent. 
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Also included in the results is a column which shows whether or not a normal distribution with 
given input parameters will be accepted (“Yes”) or not (“No”) under the corresponding plan. 

This is calculated from the inequality: 

Ll-k<7> x 

mini 

and the result indicates whether or not the inequality holds. A final column gives the critical 
value licit, which is the value for which the equality holds exactly (i.e., the minimum of the mean 
that will result in acceptance). 

The historical parameters for this problem were selected to illustrate an important cautionary. 

The same underlying distribution may lead to acceptance under a larger plan and rejection under 
a smaller plan. This is true in the example because the mean was chosen to be very close to the 
critical values for each of these plans. Note that for the (292, 2.7889) plan: 

ju-kcT = 500.12 > 500 


and for the (256,2.8045): 


// — ka = 499.96 < 500. 

Depending on the application, one might question whether such a small difference (which is 
totally an artifact of the small difference in the fc-value for the test) is meaningful. When this is 
the case, clearly larger samples on the whole will provide less conservative results. 
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